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The author explores the technology procurement process in Ontario’s publicly 
funded school districts to determine if it is aligned with relevant research, is 
grounded in best practices, and enhances student learning. Using a qualitative 
approach, 10 senior leaders (i.e., chief information officers, superintendents, 
etc.) were interviewed to reveal the most important factors driving technology 
acquisition, governance procedures, and assessment measures utilized by 
school districts in their implementation of educational technology. The data 
were transcribed and submitted to “computer-assisted NCT analysis” (Friese, 
2014). The findings show that senior leaders are making acquisitions that are 
not aligned with current scholarship, that districts struggle to use data-driven 
decision-making to support the governance of educational technology 
spending, and that districts do not have effective assessment measures in place 
to determine the efficacy of a purchased technology. The study is meant to 
serve as an informative resource for senior leaders and to present research- 
based approaches to technology procurement. 


Introduction 

Over the last 30 years, the rapid advancement of emerging educational technologies 
(ed-tech) and their subsequent consumerization has led to an unprecedented shift in the K-12 
education landscape (Bebell, O’Dwyer, Russell, & Hoffmann, 2010; Jenkinson, 2009). These 
unique teaching and learning tools have provided school districts with the opportunity to 
modernize their classrooms and equip their students with critical 21st century skills (Culp, 
Honey, & Mandinach, 2005). However, the whirlwind speed of technological development and 
its resulting impact on educational organizations’ policies, structures, and processes has created a 
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critical need for effective technology management and leadership (Bellamy, 2007). The 

responsibilities of educational leaders and policy makers are further problematized by the 

collective weakness of educational technology research and its inability to provide substantial 

empirical evidence to support technology integration as a means of improving student learning 

(Bebell et al., 2010; Jenkinson, 2009). Notwithstanding these profound challenges, Ontario’s 

publicly funded school districts have made multimillion-dollar investments in educational 

technology, with spending showing no signs of slowing (Ministry of Education, Ontario, 2014). 

It is important to note that while Ontario’s publicly funded school districts are spending 

large amounts of money on technology, they are doing so with minimal guidance from the 

province. Recently the Ontario Public School Boards’ Association (2013) voiced their frustration 

with the province’s lack of technology leadership and formal direction: 

Many other jurisdictions have moved vigorously ahead to define a vision to 
guide education well into the 21st century and we urge Ontario, which is a 
leader in student achievement and in education in so many spheres, to take up 
this challenge. . . . This is a matter of public confidence in our education 
system. Students, teachers, parents, school boards—all our education 
stakeholders—are ready to embrace this vision, (p. 1) 

This research acknowledges the frustrations of these Ontario school leaders and seeks to 

determine if the technology procurement process in Ontario’s publicly funded school districts is 

aligned with the relevant research, is grounded in best practices, and enhances student learning. 

The study examines the procurement, governance, assessment, and return on 

investment (ROI) measures utilized by school districts in their implementation of educational 

technology. The following question provided the overall direction for the research study: 

■ How do Ontario’s publicly funded school districts make decisions on 
acquiring new technology for their school systems? 
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While school boards are making increased efforts to engage their stakeholders as compared to 
years past, taxpayers are still largely in the dark in relation to technology acquisition. In order to 
address this gap in knowledge, the following questions guided this investigation: 

■ What are the most important factors senior leaders consider when procuring 
educational technology? Is this supported by relevant research? 

■ What are the governance procedures for technology procurement and spending? Is this 
guided/supported by data-driven decision-making? 

■ What kinds of assessment measures are in place to decide on the effectiveness of a 
technology and its impact on student learning? How do school districts measure and 
report on the return on this type of investment? 

To answer these questions, the researcher compares and contrasts the reasons districts purchase 
technology against its theoretical framework that emerged from a review of the literature. 

Literature Review 

This section will explore the most important facets of technology decision-making in 
education. To gain a better understanding of the emerging issues, challenges, and opportunities 
for educational leaders today, the author begins with an in-depth examination of the foundational 
literature used in this study. This will provide the reader with an understanding of what the key 
authors and bodies of research posit about educational technology acquisition. 

Conceptual and Theoretical Frameworks 

As Creswell (2013) clearly states, “We conduct qualitative research because we want to 
understand the contexts or settings in which participants in a study address a problem or issue” 
(p. 40). With stakeholders in mind, it was important that the conceptual framework for this study 
was designed to break down the key elements of effective senior-level technology decision- 
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making. Consequently, the conceptual framework was developed by consulting both academic 
and professional literature. The following areas of exploration were identified: 

1. technology procurement and spending 

2. academic impact of technology on student learning 

3. data-driven decision-making (DDDM) 

With educational technology decision-making being such a large and complex issue (still 
evolving), it proved very difficult to find a single theory that encompassed all of the relevant 
concerns and topics related to this field of study. Of growing interest is the literature surrounding 
the essential conditions to support technology implementation (Gomes, 2011; International 
Society for Technology in Education [ISTE], 2009; Krueger, 2013), the factors that affect 
student learning (Finkel, 2012; Marzano, Pickering, & Pollock, 2001; McCombs & Whisler, 
1997), and data-driven decision-making (Ikemoto & Marsh, 2007; Mandinach, Honey & Light, 
2006). 

Technology Procurement and Spending 

Total cost of ownership. This framework (TCO) draws on the work of Krueger (2013) 
and Greaves and Hayes (2008) and highlights the need for senior leaders to be cognizant of total 
cost of ownership when they go to market. Krueger writes, “To get started, you must understand 
the cost of a technology initiative over the life of the project. . . . Initial purchases, training, and 
implementation costs must be amortized or annualized, and ongoing costs must be added in” (p. 
26). 

Organizational vision. The literature draws attention to an overall district vision as 
imperative for sound technology decision-making and leadership (Fullan, 2013; Gomes, 2011; 
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ISTE, 2009). If a technology does not fit within the broader vision and goals of the school board, 
then it should not be purchased. Furthermore, the overall organizational vision should be 
informed by the entire organization (ISTE, 2009). 

Efficacious funding. In order for school districts to avoid implementation failures, they 
need to determine whether the amount of funding they have (and will have in the future) is 
enough to successfully purchase, deploy, and assess a new technology (Greaves & Hayes, 2008; 
Sundeen & Sundeen, 2013). 

Academic Impact of Technology on Student Learning 

Teacher training. For any technology implementation to have a high rate of success, the 
role of the teacher needs to be constantly considered (Avidov-Ungar & Eshet-Alkakay, 2011). In 
particular, professional learning and development for educators is universally identified in the 
literature as an essential condition for successful technology procurement and implementation 
(Culp et al., 2005; Greaves & Hayes, 2008; ISTE, 2009). 

Increased student engagement. According to Finkel (2012), aside from standardized 
test scores, student engagement is the biggest measurement districts are currently using to assess 
the academic ROI on a technology purchase. This claim is aligned with both the American 
Psychological Association’s (APA Work Group of the Board of Educational Affairs, 1997) 
learner-centered framework and Marzano et al.’s (2001) nine categories of instructional 
strategies that affect student achievement. 

Collaboration. Implementing technology in classrooms can facilitate collaboration in a 
number of ways that would have otherwise not been possible. Once again both the APA’s (1997) 
learner-centered framework (social influences on learning ) and Marzano et al.’s (2001) nine 
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categories of instructional strategies that affect student achievement ( cooperative learning ) 
mention learning through collaboration. 

Data-Driven Decision-Making 

Timeliness. In their analyses of data-driven decision making, Ikemoto and Marsh 
(2007) found that “individuals in many districts . . . commonly complained that state test data 
were not timely” (p. 120). With the Internet and technological tools providing students with real¬ 
time feedback in their personal lives (e.g., video games, text messaging, etc.), teachers and 
administrators need to be given the tools to deliver the same feedback in classrooms (Mandinach 
et al., 2006). 

Accessibility. Along with providing timely feedback, data tools need to be accessible to 
relevant stakeholders. Mandinach et al. (2006) define accessibility as “how easy the tool is to 
access and use” (p. 10). Similarly, Ikemoto and Marsh (2007) found that if the access to data is 
not easy, it will not be utilized. 

Capacity. While a school board’s IT department has access to a variety of student data 
(e.g., login statistics, academic records, etc.) and can analyze the data to inform decision-making, 
teachers and administrators lack that ability. In order for quality data collection to occur, districts 
need to either pursue technologies that build the district’s technical capacity to analyze/use data 
in their decision-making or ensure that personnel are being put in place to analyze the data 
emerging from classroom technologies (i.e., data coach ; Ikemoto & Marsh, 2007). 
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Theoretical Framework of Technology Decision-Making 

Based on the literature review of existing research on technology decision-making, a 
research model has been developed that is aligned with the study’s conceptual framework. The 
research model is used as a basic expectation of senior leaders who are making technology 
decisions at the district level (see Figure 1). 

Research Design and Methodology 

The aim of this research was to explore the decision-making process behind technology 
procurement in Ontario’s publicly funded school districts. This study employed a qualitative case 
study approach using both face-to-face and online interviews, which allowed the participants in 
the study to describe their experiences in context, so that the research community and 
stakeholders are better positioned to understand their actions (Robottom & Hart, 1993). Although 
educational technology research is dominated by qualitative studies (Bebell et al., 2010), 
technology spending, its academic impact, and data-driven decision-making have not been 
studied in combination to determine how senior leaders make educational technology decisions. 
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Figure 1. Theoretical Framework of Technology Decision-Making. 


Selection of Participants 

Participants were selected using purposeful sampling in which, as Creswell (2013) 
writes, “the inquirer selects individuals and sites for study because they can purposefully inform 
an understanding of the research problem and central phenomenon in the study” (p. 156). 
Therefore, it was important to select individuals who are in leadership positions and make high- 
level decisions regarding school technology acquisitions, assessments, and analyses. These 
individuals typically hold the title of chief information officer” (CIO) or superintendent and 
possess the status, experience, and knowledge required for this study. It was decided that the 
study would need at least 10 senior leaders to participate for the findings to be impactful and 
once the tenth interview had been scheduled the researcher no longer accepted participation 
requests. 
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Instrumentation 

For the exploration of the central phenomenon of this study, a semistructured interview 
design with mostly open-ended question was deemed appropriate. The questions were developed 
based on the Theoretical Framework of Technology Decision-Making, the research questions, 
and current technology trends/topics that were emerging in both professional and general 
discourse. The researcher conducted the interviews over a four-month period and then 
subsequently transcribed the audio recordings. Validation was sought from participants through a 
member check as well (Creswell, 2013; Tracy, 2010). 

Data Processing and Analysis 

The analytic approach that was used in this research study is “computer-assisted NCT 
analysis” (Friese, 2014) using ATLAS.ti, a computer-aided qualitative data analysis software 
(CAQDAS). Computer-assisted NCT analysis is adapted from Seidel (1998) and is comprised of 
three basic components, which are noticing things, collecting things, and thinking about things. 
These three basic elements are common to a large range of analytical practices in qualitative 
research and Creswell’s (2013) five research traditions in particular. 

The first phase of the NCT method required the researcher to notice interesting things 
in the data and write down notes or attach preliminary codes. The second phase involved the 
researcher collecting similar items under existing, new, or merged code labels. Although NCT 
analysis does not prescribe any particular way of coding, the researcher of this study employed 
the use of both deductively and inductively developed codes (Friese, 2014). The final phase 
engaged the researcher in deeper thinking after coding to discover patterns and processes across 
the cases. Additionally, participant responses were tallied and summed for particular questions 
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based on frequency of mention. It was important to use the NCT method in this research study in 
order to use the full functionality of ATLAS.ti in analyzing the data (e.g., Phase 2 of the NCT 
method is best accomplished through the use of ATLAS.ti’s code co-occurrence table tool). 

Findings 

The 10 participants were senior-level decision-makers who currently hold leadership 
positions in publicly funded school districts in Ontario. They are identified by the pseudonyms of 
Amanda, Anthony, Clive, Daniel, Gabriela, Julia, Megan, Nicholas, Stan, and Walter. Table 1 
provides a description of each senior leader and his or her respective school district. 

Most Important Factors Driving Technology Acquisition 
The data revealed that the 10 senior leaders considered many of the same factors when 
acquiring technology for their school districts. In discussing the process, they all acknowledged 
numerous challenges in determining what technologies to purchase for their stakeholders. During 
the interviews, participants were asked to identify and rank the most important factors they 
consider when procuring technology for their school districts. In reviewing the data, the most 
common response (in terms of frequency) was (1) cost-related factors, which was mentioned by 
all 10 senior leaders. Next, (2) impact on school board infrastructure was considered by nine of 
the participants. Seven senior leaders deliberated (3) product specifications, and four measured a 
product’s (4) alignment with technology plan/organizational vision. Finally, three participants 
considered (5) the impact of technology on instruction and student learning when purchasing 
hardware/software. Table 2 shows the participants’ responses. 


10 



Educational Technology Decision-Making: Technology Acquisition for 746,000 Ontario Students 


Table 1 

Description of Senior Leaders and Their School Districts 


Pseudonym 

Position 

Main 

responsibilities 

No. of 
students 

No. of 
teachers 

No. of 
schools 

Size 

Amanda 

Superintendent 

Academic 

Under 

50.000 

Under 

3.000 

50 to 

100 

Medium 

Anthony 

Superintendent 

Academic 

50,000 to 
100.000 

3,000 to 
6.000 

100 to 
200 

Large 

Clive 


Nonacademic 

Above 

100.000 

Above 

6,000 

Above 

200 

Very large 

Daniel 

Superintendent 

Academic 

Above 

100,000 

Above 

6,000 

Above 

200 

Very large 

Gabricla 

/ 

Academic 

50,000 to 
100.000 

Above 

6,000 

100 to 
200 

Large 

Julia 

Superintendent 

Nonacademic 

Under 

50,000 

Under 

3,000 

50 to 

100 

Medium 

Megan 

CIO 

Nonacademic 

Under 

50,000 

Under 

3.000 

50 to 

100 

Medium 

Nicholas 

CIO 

Nonacademic 

50,000 to 
100,000 

3.000 to 
6,000 

100 to 
200 

Large 

Stan 

CIO 

Nonacademic 

50,000 to 
100,000 

Above 

6,000 

100 to 
200 

Large 

Walter 

CIO 

Nonacademic 

Under 

50,000 

Under 

3,000 

50 to 

100 

Medium 


l Vote. Clive and Gabricla's position titles were purposely not disclosed because revealing them could 
potentially identify them to readers. 


Cost-Related Factors 

All 10 participants stressed the importance of various cost-related factors when 
considering purchasing new technology resources for their school districts. Nine participants 
emphasized (a) affordability/price, seven discussed (b) sustainability, and two mentioned (c) 
cost-saving potential. 

Affordability/price. Participants provided several examples where high prices were a 
prominent factor and deterred their organizations from procuring certain software/hardware. 

Cost is really important and sometimes cost will put things out of our reach. 

So a one-to-one (1:1) project would be out of our reach because sheer cost 

would put it out of reach. (Amanda) 
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Sustainability. Seven school leaders reported that the sustainability of a new technology 

needed to be considered before it is purchased. 

So affordability and sustainability are absolutely key. As you are probably 
aware, we recognize that the cost of acquisition is often just the smallest 
component of overall technology costs. (Megan) 

Cost-saving potential. Participants were asked if they were more prone to want to acquire cheap 

or free technology/technological tools. The wariness that cheap/free technology would not 

actually save the organization money in the long term figured very heavily in participants’ 

responses. In fact, the majority of participants did not consider the resources “free” at all. 

Free is not necessarily “free.” I can get free software and then I pay for all the 
support, and all the training, and all of the maintenance, and all of the 
upgrades. I could pay for something upfront and the company pays for the 
maintenance, the support, and the upgrades. Your call. (Gabriela) 


Impact on School Board Infrastructure 

Of almost equal importance to cost-related factors was the impact a new technology 
would have on the school board’s infrastructure. Five participants emphasized (a) 
access/mobility and four discussed (b) compatibility with the existing network. 

Access and mobility. Several participants described their current focus on providing all 
stakeholders with access to resources and core services through devices. The most common way 
to do this was through increasing the district’s Wi-Fi presence to allow stakeholders the ability to 
access online content. 

After deciding that we significantly needed the infrastructure, we then bought 
routers, switches, and wireless technology in an effort to provide an 
environment that was ready for ubiquitous access. (Stan) 
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Compatibility with the existing network. Four of the participants acknowledged that 
compatibility with their organization’s existing network (i.e., wireless infrastructure, human 
resources, etc.) was very important and minimized the amount of changes that would occur 
during implementation. 

Compatibility with our existing network is a huge, huge factor. Probably 
number one . . . that it would fit in with our existing technology, our existing 
infrastructure and not involve large changes. (Amanda) 

Product Specifications 

In comparison to the previous two factors, product specifications were less apparent in 
the data when senior leaders described technology acquisition. However, seven participants did 
state that one of the important considerations they make in the purchasing process surrounds 
what the product does and whether it meets the criteria of the organization. Specifically, five 
senior leaders identified (a) durability, three mentioned (bl) functionality, and three named (b2) 
assistive features as key specifications they evaluate when acquiring technology. 

Durability. Five senior leaders accounted for the importance of durability up front. 
Walter explained that purchasing a product that is known to be durable might require his district 
to pay a slight premium for that technology. He went on to speak about how a difference of a few 
thousand dollars between competing technologies could mark the difference between one 
product failure over 8 years and 2,500 failures. 

Functionality. Three participants expressed that functionality was a key factor they 
assessed when acquiring new technology. Julia highlighted the importance of functionality from 
the very beginning of the interview. 

So probably the single most important factor when you are making these 
decisions is “Does the actual delivered functionality meet the requirement?” 

. . . Really we look at it pretty dispassionately. (Julia) 
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Assistive features. Three participants expressed that assistive features were key 
capabilities they looked for in the products they evaluated. Walter provided an example of how 
his district managed to address the needs of special education students in the procurement 
process. He described that if he were looking to acquire 1,000 laptops he would automatically 
ensure that 3% (or 30 units) came with a larger display and touch screen functionalities to 
accommodate those students with visual impairments. 

Alignment With Technology Plan/Organizational Vision 

Five participants made reference to overarching (formal) documents that were actively 

guiding their current and future technology acquisitions. Four participants reported that their 

districts had technology plans that had been developed internally, while one participant’s 

organization had adopted an external technology standard. Gabriela outlined her efforts in 

writing and publishing a technology plan that was well received throughout the district. 

The current iteration of things is I wrote and published a technology plan 
amidst the district 3 years ago. We just finished year two of it. It is a 5-year 
plan and the plan is on our website and it is all about going mobile and in a 
culture of accessing mobility. (Gabriela) 

However, even with a very clearly laid out technology plans, Stan and Megan both 
outlined that in such large organizations there were going to be stakeholders that deviated from 
the vision. 

So of course with [several hundred] schools we are going to have the 
“renegades,” for lack of a better term, who want to go out and try something 
different and new. (Stan) 

While Stan’s department responds by evaluating the situation more closely and by advising the 
administrator of the potential risks and challenges ahead, Megan was less optimistic about 
getting the staff in her school district to understand the proposed vision. 
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Because we have a problem where I would say . . . the majority of the system 
does not care what the [technology plan] says. They have their own vision of 
how technology should be used and what technology they need. (Megan) 


Impact of Technology on Instruction and Student Learning 

Student learning was discussed in all of the senior leader interviews. However, only 

three participants mentioned its prominent role in the acquisition of a new technology. Two 

senior leaders placed emphasis upon the technology’s ability to improve (a) pedagogy and one 

considered (b) student learning in their organizations when evaluating a potential acquisition. 

Pedagogy. In addition to other factors they previously mentioned, Gabriela and 

Anthony focused on supporting the needs of their teachers so that instruction throughout their 

school boards could be both improved and adequately supported. 

We do not go out and buy gear because it is the “equipment du jour.” We go 
to meet the pedagogical needs of the teacher ... to meet the pedagogical 
needs is the primary piece in there. (Gabriela) 

Pedagogy is first. You can put in all of the technology in the world but if the 
type of instruction or strategies being used by teachers is not effective, it does 
not matter what the technology is. (Anthony) 

Student learning. Nicholas outlined how his district sought to improve student learning 
through technology. For him the ISTE standards his organization adopted give his department 
solid criteria from which to assess potential acquisitions. He mentioned that since the province of 
Ontario lacked a central curriculum or mandate around student learning and technology, the 
board found guidance through ISTE. 

And they have what they call the “ISTE Strands.” Six strands that, again, I 
don’t know if you have seen them . . . but they deal with how to use 
technology for learning. So we saw a perfect fit with ISTE on how to integrate 
technology into learning. (Nicholas) 
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Governance and Data Support 

One of the aims of this research study was to identify the governance procedures for 
technology procurement and spending in publicly funded school districts. Analysis of the 
interview data uncovered that policy, committees, and senior district leadership play important 
roles in the governance models used to support technology acquisition. Moreover, it was 
determined that while senior leaders and their districts are making varied (and inconsistent) uses 
of data in their organizations, most struggled to use data in meaningful ways (i.e., to support 
spending, to make in-class decisions, and so on). 

Policy 

Nine participants acknowledged their obligation to abide by their district’s (a) internal 
purchasing policies, while only five referenced the province’s (b) Broader Public Sector (BPS) 
Procurement Directive. 

Internal purchasing policies. Most of the participants recognized the importance of 
having/creating internal purchasing policies that outlined their expectations during the 
acquisition process. It should be noted that the school board purchasing policies most senior 
leaders described were not specific to technology and often extended to all board acquisitions 
(e.g., furniture, equipment, etc.). 

We have got to go through a process of getting competitive quotes. So it is 
fairly rigorous from the technology standpoint but also from the purchasing 
standpoint that would apply to whether we are talking about technology or 
buying garbage bags. (Megan) 

Broader Public Sector Procurement Directive. Five participants described the 
importance of ensuring the acquisition process follows the guidelines set out by the government 
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of Ontario. Clive described how the main reason his organization respects the formality of the 

directive is because of the auditing process they undergo each year. 

Anything that we buy has to fit into that Broader Public Sector Procurement 
Guidelines. The reason for that is also we do get audited every year. . . . All 
the POs are cut and everyone else has looked at it. “How was this approved? 

Who approved it?” (Clive) 

Committees 

The data showed that anywhere from one to four different committees could be 
involved in a single acquisition. In most cases the committees featured representation from both 
academic and business stakeholders. Eight participants referred to (a) the role of teachers, six 
mentioned (b) the role of academic consultants, and five spoke about (c) the role of IT as central 
components of the procurement process. 

Role of teachers. Four participants described formal programs/roles their school 
districts had created to engage teachers in the use, purchase, and evaluation of the technology 
being brought into the classroom. In contrast, four other participants reported informal ways 
teachers were driving technology decision-making. Clive outlined that his board methodically 
selected 60 teachers (from both elementary and secondary schools) to hold leadership positions 
related to digital learning. These teachers maintained their full-time role in the classroom but 
were also called upon to run pilots, give feedback, and engage with the district at committee 
meetings. In contrast, Daniel and Stan reported that their teachers were supporting 21st century 
learning on their own. However, Megan expressed that some teachers and schools had become 
very resistant to integrating technology and that this took away from the overall goal of 
improving student learning. 
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It really at some points becomes debilitating because people are losing sight 
of the goal. They are losing sight of the direction because they all come from 
their respective camps and that is what they want to achieve. (Megan) 

Role of academic consultants. Six senior leaders made reference to the role of their 

district’s academic consultants. Most of these individuals were former teachers or administrators 

who were highly involved in evaluating the technologies being considered for purchase, training 

teachers, and gathering feedback/research on technology use in the classroom. 

So we have those folks that are responsible to be the bridge or the glue 
between the technology and the pedagogy. (Walter) 

Role of IT. Five participants stressed the significance of involving the district’s IT 
technicians in the acquisition of a new technology. For many years, IT staff operated under the 
same guidelines day in and day out, and the acquisition of hardware/software was their 
responsibility alone. Today, most senior leaders communicated that the IT staff does not have 
this same level of control any longer. 

That is the shift where IT departments are no longer calling the shots. We are 
facilitating kids’ learning and teachers’ learning. And I think that has 
happened and I have seen it unfold in front of my eyes over the last six years 
or so. (Gabriela) 


Senior District Leadership 

In talking with participants about their school district’s budget, all 10 revealed that no 

other group had as much control as the board of trustees. As publicly elected officials, their role 

is to ensure that taxpayer dollars are being spent wisely and that the formal policies in place are 

in support of improved student achievement (i.e., higher report card grades, improved 

standardized test scores, and so on). Stan emphasized that, although his district’s trustees are 

ultimately focused on the dollars being spent, they also weigh in on programming decisions. 

More so we focus with our trustees, definitely on dollars, absolutely . . . but 
we also focus a lot with them on program and the integration of technology 

19 



Educational Technology Decision-Making: Technology Acquisition for 746,000 Ontario Students 


and program. . . . They are as interested in the dollars and cents as they are in 
the effect it has on students. (Stan) 

In contrast, three participants expressed that the trustees in their boards were not 

involved in the selection of devices. Their departments alone were able to make the call as to 

what technologies they were going to pursue. 

But the daily acquisitions, like choosing the tablet, the trustees did not have 
any influence on what tablet we went with or what laptops we chose to go to 
as an enhancement in schools. . . . (Anthony) 


Data Support 

The participants were asked if they employed the use of data-driven decision-making, 
or DDDM, to support their acquisition decisions. Most senior leaders claimed that their 
organizations were data driven but had difficulty explaining what types of data they were using 
to support their procurement of technology. Furthermore, three participants highlighted 
numerous challenges associated with conducting data analysis and its overall use in the decision¬ 
making process. According to the frequency in the interviews, three senior leaders discussed 
DDDM as a means (a) to drive organizational efficiency and two mentioned its ability (b) to 
drive student achievement. 

Driving organizational efficiency. Walter, Megan, and Clive all provided examples of 

data’s role in the business operations of their school districts. The level of expertise/knowledge 

they communicated on the subject was impressive. Clive provided an example of the impact 

DDDM can have on a large-size organization and the astounding potential for cost saving. 

Last year we had 26,000 calls come into our help desk for password resets and 
that is almost like 10% of the calls. . . . Each password reset is costing me $5 
so that is almost $130,000 being spent on that. Can I now do something so I 
can lower my cost there? Now money and resources are free to do something 
else. (Clive) 
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Driving student achievement. The data revealed that Education Quality and 

Accountability Office (EQAO) scores were the most common data set being used by senior 

leaders to justify or tailor technology acquisitions (and the resulting rollout). Stan admitted that 

his board did not have a good grasp on the big data movement and questioned whether they had 

the capacity as an organization to take advantage of incorporating DDDM in the near future. 

My concern is that we can become completely bogged down in data and not 
look at what is right in front of us. ... I think there are cases where it is 
obvious that the data is clear enough that you need to make this type of 
change. . . . Students want to bring their own device, so let them bring their 
own device. (Stan) 

Assessment and Return on Investment Measures 
Another aim of this research study was to determine what kinds of assessment and 
return on investment or ROI measures districts used to decide on the effectiveness of a 
technology and its impact on student learning. The 10 senior leaders made reference to a number 
of different testing methods they employed after acquiring a new technology. Data analysis 
revealed that school boards had more success conducting technical assessments on 
hardware/software than academic assessments of the technology’s impact in the classroom 
(where they had great difficulty). Furthermore, when asked how they reported the return on 
technology investments within their district, none had a formal process in place to justify their 
spending. 

Technical Assessments 

Seven senior leaders referred to the technical assessments they use throughout their 
school districts to ensure the technologies they have purchased are functioning properly and are 
being used by stakeholders. Much of the participant responses surrounded the measuring of 
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device usage (i.e., login statistics). Amanda found that the technical assessments her team carried 
out were relatively straightforward. 

So on the tech side of it, it is really pretty simple because we can see uptake of 
use. . . . We use our help desk system because we can see the certain kinds of 
breakdowns, certain products we will not use anymore, those sorts of things. 

(Amanda) 

Academic Assessments 

All 10 participants expressed that their districts were experiencing difficulties in 
conducting high-quality academic assessments that evaluated technology’s impact on their 
students’ learning. The assessments they currently used were largely anecdotal in nature. Most of 
the participants’ responses highlighted “pre” and “post” evaluations of student engagement 
measures (i.e., increased levels of attendance) during pilot studies. Stan described the difficulties 
in establishing reliable metrics to measure the academic value of the technologies they were 
acquiring. 

The majority of the data is anecdotal. No doubt about it. It is really tough to 
measure and it is a question we get all the time. (Stan) 

Given the difficulties they were having in establishing reliable metrics around technology and 
student achievement, many senior leaders opted to consult EQAO scores. However, most 
participants reported that they did not play a large role in the acquisition of a new technology; it 
was simply one of the only metrics they had available. 

Return on Investment 

Participants were asked how they reported the return on educational technology 
investments within their school districts and communities. While many senior leaders reported 
back to the board of trustees to update them on the progress of various technology deployments, 
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none had a formal ROI process in place. Once again softer reports with limited data were used to 

justify their technology acquisitions. Nicholas expressed the immense difficulty in being able to 

conduct an accurate ROI analysis in his school district. 

How do you quantify it right? ... So as we have said there are not a lot of 
ROI tools. . . . How do I know anything is improving instruction? Whether it 
is improved lighting or air conditioning. (Nicholas) 

Anthony described the disparity between the ROI measurements his department valued and the 

values of the board of trustees in his district. 

From a board of trustees’ perception, it is graduation rates and EQAO. 

Although we talked about knowing your students, knowing them at such an 
intimate level, their learning needs, that is not what gets shared in the 
boardroom. (Anthony) 


Discussion 

The findings revealed that cost-related factors are the most important concerns senior 
leaders have when acquiring a new technology for their school district. While the importance of 
financial considerations is echoed throughout the literature (Greaves & Hayes, 2008), they 
should not be driving technology acquisition. This finding also echoes the results of Morrison, 
Ross, Corcoran, and Reid’s (2014) recent study of ed-tech purchasing in the United States. They 
write, “The most frequent challenge expressed in open-ended survey responses, and most 
strongly emphasized by superintendents, related to funding and financial concerns” (p. 5). 

The finding that product specifications weigh heavily in participants’ acquisition 
decisions is not explicitly noted in the literature. When describing the most costly mistakes 
associated with technology procurement, Greaves and Hayes (2008) and Krueger (2013) cite 
hardware and software problems as one of the most frequent occurrences in American school 
districts. The participating school leaders seemed to recognize this and are taking proactive 
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measures to ensure they are receiving the best financial return on their purchases. However, this 
heavy focus on product specifications only further extends the notion that district leaders are 
squandering opportunities to make real connections between technology and learning. 

A few participants recognized the importance of their district’s technology 
plan/organizational vision. However, none noted that the overall vision had been informed by 
the entire organization. ISTE (2009) implores senior leaders to utilize, “proactive leadership in 
developing a shared vision for educational technology among all education stakeholders, 
including teachers and support staff, school and district administrators, teacher educators, 
students, parents, and the community” (para. 1). Perhaps the absence of this meaningful 
engagement is why some leaders are having difficulty getting their stakeholders to buy into their 
technology plans/policies. 

An emphasis and focus on technology’s impact on instruction and student learning 

during acquisition is strikingly limited in the data collected from senior leaders. This is highly 

inconsistent with the current literature on supporting student learning through technology and 

this study’s conceptual framework. Without a focus on pedagogy and student learning, school 

leaders and their districts risk being exposed to numerous challenges (Fullan, 2013). 

The findings revealed that committees also largely contributed to the governance of 

educational technology acquisition and implementation in the participating school districts. It 

was apparent that all of the districts are in a significant period of transition and have recently 

undergone changes to better address the technology needs in their organization. This is consistent 

with recent research emerging from Ontario. Clarke et al. (2014) write: 

School districts are investigating and implementing new organizational 
structures that embed coordinated approaches to using technology. They are 
in the early stages of aligning departments and jurisdictional responsibilities 
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to enable system-wide approaches to 21st century teaching and learning (e.g., 
establishing working relationships between IT and curriculum), (p. 19) 

While most of the senior leaders mentioned the key roles teachers play on certain committees or 

in the rollout/implementation process, only one discussed their direct involvement in the 

acquisition of a new technology. As mentioned previously, this contradicts the reviewed 

literature that says if teachers are given a greater voice in the decision-making process, the 

success rate of the technology implementation greatly increases (Bill and Melinda Gates 

Foundation, 2014). 

It is important to note that, in the absence of effective academic assessment measure, 
some districts are using EQAO scores when making technology-related decisions. This practice 
directly contradicts the reviewed literature that stresses standardized tests only measure fact 
recall and broad knowledge in students and are largely unrelated to the 

development/measurement of 21st century skills (Bebell et al., 2010). Perhaps the role of EQAO 
test results in Ontario education needs to be revisited. 

Finally, the finding that participating school districts had no formal ROI process in 
place is consistent with very recent research on technology acquisition and reinforces the notion 
that districts need to shift their focus to analyzing technology’s impact on student learning. 
Krueger (2013) advocates for a redefinition of return on investment that focuses less on cost- 
related factors and more on student learning and community-building (i.e., developing 21st 
century skills, engaging parents, etc.). 


Implications 

The author demonstrated that when compared and contrasted against the study’s 
Theoretical Framework of Technology Decision-Making (refer back to Figure 1), senior leaders 
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are not making decisions that are aligned with formal research, grounded in best practices, and in 
enhancement of student learning. It has become increasingly clear that districts lack guidance 
when purchasing technology. However, it is the author’s stance that using relevant scholarship as 
one’s strategy during technology decision-making can help districts avoid missed opportunities 
and costly mistakes. The framework was renamed the Strategic Model for Technology 
Acquisition (Figure 2) and gives districts the opportunity to ensure that each decision they make 
is aligned with the key theories and body of research. For the purposes of this framework the 
areas were renamed strategic academic assessment (TIC), strategic data use (TAC), and strategic 
acquisition (TOE). 

While the results of this study produced similar results to the recent studies conducted 
on educational technology procurement in school districts in the United States, the Canadian 
context should be further explored. Little research has been discovered on educational 
technology acquisition in Canada, and this study should be accompanied by investigations into 
other related areas. One cannot assume similarities between the U.S. and Canadian education 
systems based on the results of this research. 
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Figure 2. Strategic Model for Technology Acquisition. 

Conclusion 

While it may seem that districts are encountering numerous challenges in several key 
areas of educational technology procurement, there is one area in which they excel: purchasing. 
From durability to sustainability, affordability to cost-saving, district leaders are going above and 
beyond to ensure they are receiving the best deal possible from vendors. So what is the problem? 
The issue lies with the fact that school leaders have become so focused on not integrating 
technology for technology’s sake, that they are ignoring/undervaluing the pedagogy and the 
product’s impact on student learning. This is misguided and highly dangerous when you consider 
the price districts in the United States have had to pay for taking the same approach (i.e., poor 
usage rates, millions of dollars wasted, and so on). 
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The researcher implores Ontario’s Ministry of Education to take greater leadership in 
the area of technology acquisition. With districts feeling pressured by stakeholders to quickly get 
technology into their classrooms and the competition for provincial funding growing steadily, the 
roles and responsibilities of senior leaders are becoming increasingly complicated. The 
challenges highlighted in this study are not isolated but systemic and need to be quickly 
remedied. One potential solution lies in the Ministry of Education, which can provide the support 
and guidance needed to implement meaningful education reform in Ontario. The rapid pace at 
which technology is advancing is certainly not helping their case. Yet while the province is right 
in its reluctance to chase technology for its school systems, it should instead be sprinting towards 
establishing best practices and leveraging personalized learning opportunities for its students. 
After 15 years in the 21st century Ontario has been sluggish in that regard. In order to create an 
education system rich in pedagogy and empowered by 21st century technologies, provincial 
leadership needs to better support its school district leaders. 
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